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Abstract — In the matter of selection of sample time points for 
the estimation of the power spectral density of a continuous time 
stationary stochastic process, irregular sampling schemes such 
as Poisson sampling are often preferred over regular (uniform) 
sampling. A major reason for this preference is the well-known 
problem of inconsistency of estimators based on regular sampling, 
when the underlying power spectral density is not bandlimited. 
It is argued in this paper that, in consideration of a large sample 
property like consistency, it is natural to allow the sampling rate 
to go to infinity as the sample size goes to infinity. Through 
appropriate asymptotic calculations under this scenario, it is 
shown that the smoothed periodogram based on regularly spaced 
data is a consistent estimator of the spectral density, even when 
the latter is not band-limited. It transpires that, under similar 
assumptions, the estimators based on uniformly sampled and 
Poisson-sampled data have about the same rate of convergence. 
Apart from providing this reassuring message, the paper also 
gives a guideline for practitioners regarding appropriate choice 
of the sampling rate. Theoretical calculations for large samples 
and Monte-Carlo simulations for small samples indicate that the 
smoothed periodogram based on uniformly sampled data have 
less variance and more bias than its counterpart based on Poisson 
sampled data. 

Index Terms — spectrum estimation, periodogram, regular sam- 
pling, Poisson sampling, consistency, rates of convergence. 



I. Introduction 

Estimation of the power spectral density of a continuous 
time wide sense stationary stochastic process is an old prob- 
lem. A set of regularly (uniformly) spaced samples is generally 
used for this purpose. When the process is bandlimited, the 
spectral density of the original process can be recovered from 
that of the sampled process, provided that the sampling is fast 
enough. In such a case, estimation of the spectral density from 
finitely many observations at an appropriate sampling rate is 
a well established topic and many useful nonparametric and 
parametric methods have been developed |[T| . If the underlying 
process is not bandlimited, the spectral density of the original 
process is not identifiable from regularly spaced samples, 
because of the problem of wrapping around of the spectral 
density caused by the process of sampling - also known as 
aliasing |2|. In such a case, one cannot estimate the spectral 
density consistently from regularly spaced samples at any fixed 
sampling rate. 
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This inadequacy of sampling at regular intervals necessitated 
the exploration of other strategies for sampling. Shapiro and 
Silverman (j3] considered alias free sampling schemes in the 
sense that two continuous time processes with different power 
spectra do not produce the same spectrum of the sampled 
sequence. They proved that additive random sampling through 
a class of renewal processes, including the homogeneous 
Poisson process, is alias free. Beutler [4| formalized the 
definition of alias free sampling relative to a family of spectral 
distributions and gave different sampling schemes that are 
alias free relative to different families of spectral distributions. 
Masry [5 1 provided a modified definition of alias free sampling 
that would guarantee the existence of a consistent estimator of 
the power spectral density. 

Subsequently, the possibility of breaking free from the 
nuisance of aUasing through irregular sampling enthused 
many researchers and practitioners. Masry ||6l proposed a 
Poisson sampling based estimator similar to the smoothed 
periodogram, and proved that the proposed estimator is con- 
sistent for any average sampling rate, under certain conditions. 
Several irregular sampling based methodologies of estimation 
of the power spectral density of a non-bandlimited process 
have also been proposed Q- HJl and applied to various fields 
including signal and image processing. Some of these methods 
are analogous to methods developed for regularly sampled 
data. 

Irregularly spaced data occur naturally in many practical 
situations like seismic data lfT2l . turbulent velocity fluctuation 
[131, Laser Doppler Anemometer (LAD) data lfT4l . wide-band 
antenna arrays |15l. Computer Aided Tomography, Spotlight- 
Mode Synthetic Aperture Radar [16] and so on. There have 
been attempts to use standard methods for spectrum estimation 
for such data, after suitable weighting fTT\ or interpolation ifTJl 
of the data. The advent of new methodologies for irregularly 
sampled data are important for such problems. 

However, in many appUcations, including internet traffic 
data lHU, seismology |fT9l. llol. image processing flT] and 
so on, one has control over the sampling mechanism. In 
such cases, the selection of the sampling scheme is a serious 
issue. Regular sampling is easier to implement than irregular 
sampling. The literature on power spectrum estimation based 
on regular sampling contains a large collection of methods, 
and these have been studied in detail. An estimator based on 
regularly sampled data is generally computationally simpler 
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than a similar estimator based on irregularly sampled data. 
Moreover, it is well known that spectral estimators based 
on irregularly sampled data have higher variances than those 
based on regularly sampled data 1221 . Il23l . On the other hand, 
the possibility of aliasing and the proven inconsistency of 
spectral estimators based on regular sampling are arguments 
in favour of using irregular sampling. 

These opposing arguments necessitate a thorough com- 
parison of spectral estimators based on different sampling 
strategies. A systematic numerical comparison is not available 
in the literature. A simulated example given by Masry I24l was 
meant only to highlight the problem of aliasing and to demon- 
strate how it can be overcome with irregular sampling. Studies 
by Roughan |18 | in the special case of active measurements 
for network performance produced mixed results, which led 
the author to conclude that, while spectral estimators based on 
Poisson sampling have less efficiency (i.e., high variance), such 
techniques could be used to detect periodicities in the system, 
and to determine which rate of regular sampling would be 
inadequate. In the absence of a comprehensive empirical study, 
it appears that many researchers shun regular sampling mainly 
because of the stigma of inconsistency attached to spectral 
estimators based on regularly sampled data |25|. 

Consistency of an estimator concerns its behaviour as the 
sample size goes to infinity. However, it does not make 
practical sense to let the sample size tend to infinity with 
fixed sampling rate. If one gathers more and more resources 
to increase the sample size, one can use some of these 
resources to sample faster. Realizing this, practitioners fix the 
intended range of spectrum estimation, and then sample an 
appropriately filtered process at a sufficiently high frequency 
to avoid aliasing. Sometimes one goes for successively higher 
rates of uniform sampling to determine an appropriate rate of 
sampling |21|. However, these common sense approaches are 
yet to be backed up by appropriate asymptotic calculations. 
There is a need to bridge this gap by working out the 
large sample properties of estimators when the sampling rate 
changes suitably as the sample size goes to infinity. 

Let (/)(•) be the spectral density of a mean square continuous, 
wide sense stationary process {X{t), —oo < t < oo}. The 
most commonly used nonparametric estimator of (/){■) based 
on the n uniformly spaced samples at the sampling rate p is 
(ppi-), where 

^p(A) = ^ 7pM^(^"'^)e"'^l[-Trp,7rp](A), (I.l) 

\v\ <n 

where K{-) is a covariance averaging kernel, 1a (A) is the 
indicator function that takes the value 1 when X E A and 
the value otherwise, {bn} is a sequence of window widths 
such that bn and n6„ — > cx) as n cx) and ^p{v) is an 



estimator of the covariance function, defined as 

V = — (n— 1),— (n — 2),...,n — 2,n — 1. 

This estimator is known to be consistent when the underlying 
process is bandlimited. 

In the present work, without assuming that the process 
is bandlimited, we examine the asymptotic properties of the 
estimator given in (II. Il l by letting p go to infinity at an 
appropriate rate, as n goes to infinity. In the sequel, we 
shall use the notation p„ instead of p, in order to explicitly 
indicate the dependence of the sampling rate on the sample 
size. Accordingly, we use the following modified notation of 
the estimator of ( 11.11 ): 

4>nW = 77— y2 7«(«)-^(fenf)e"~l[_^p„.^p„](A) 
\v\<n 

(1.2) 

where 

n p[ \PnJ \ Pn J 

V — —{n — l),—{n — 2), . . . ,n ~ 2,n — 1. 

(1.3) 

In Section |II] we prove the consistency of the estimator 
(/)„(•) under some general conditions. In Section |III1 we 
calculate the rate of convergence of the bias and variance of 
this estimator, and determine the optimal rates at which p„ 
and nbn should go to infinity so that the mean square error 
has the fastest possible rate of convergence. Subsequently, we 
compare the rates of convergence of the bias and the mean 
squared error (MSB) of this estimator with those of a similar 
estimator proposed by Masry f6l, based on non-uniform (Pois- 
son) sampling. We present the results of a simulation study in 
Section|IV]and provide some concluding remarks in Section [Vl 
The proofs are given in the appendix. 

II. Consistency of the estimator 

Consider the mean square continuous, wide sense stationary 
stochastic process {X{t), — (x < t < oo} with zero mean, 
(auto-)covariance function C(-) and spectral density (p{-). In 
order to prove that the estimator ( 11.21 ) is consistent, it is 
sufficient to show that the bias and the variance of the estimator 
tend to zero as the sample size (n) tends to infinity. 

We assume the following condition on the covariance func- 
tion C(-). 

Condition 1. The function ho{-), defined over the real line as 

ho{t) ~ supg>|(| |C(s)| is integrable. 

Remark 1. Condition 1 is equivalent to saying that the 
covariance function C(-) is bounded over [0, cxd) by a non- 
negative, non-increasing and integrable function. 
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We assume the following conditions on the choice of the 
kernel K{-), the kernel window width b„ and the sampling 
rate p„. 

Condition 2. The covariance averaging kernel function K(-) 
is continuous, even, square integrable and bounded by a 
non-negative, even and integrable function having a unique 
maximum at 0. Further, K{0) = 1. 

Condition 3. The kernel window width is such that 6„ — J- 
and nbn — oo as n — > oo. 

Condition 4. The sampling rate is such that p„ oo and 

Pnbn — > as 71 — ^ OO. 

Remark 2. Note that the estimator 0„(-) can be written as 
1 



satisfies 



Pnbn 



Init^)Wn ( ] dfi, 

Pnbn 



t=i ^P"' 



2'Knpn 

oo /'OO 



lr-71 



](A), 



where 

/n(A) = 

W„(A) = 

j = -oo ■ 

Thus, 0„( ) is the smoothed version of /„(•), the periodogram, 
where the degree of smoothness is controlled by the smoothing 
parameter p„6„ of the frequency domain window Wn{ )- 
Condition 4 says that this parameter goes to zero, and the 
sampling rate goes to infinity, as the sample size goes to 
infinity. 

Theorem 1. Under Conditions 1^, the bias of the estimator 
4>n{-) given by f |/.2| ) tends to zero uniformly over any closed 
and finite interval. 

Before examining the variance of the estimator we assume 
the following condition on the fourth order moments of the 
process {X{t)^ — oo < t < oo}. 

Condition 5. The fourth moment E\X{t)\'^ exists for every 
t, and the fourth moment function E[X{t)X{t + vi)X{t + 
V2)X{t + vs)] is a function only of the lags vi, V2 and 
V3. Further, the fourth order cumulant function Q{vi,V2,V3), 
defined by 

Q{vi,V2,V3) = P(W1,1'2,W3) - Pg {.Vi,V2,V3), 

where 

P{vi,V2, V3) = E{X{t)X{t + v,))X{t + V2)X{t + V3) 
and Pg{vi,V2, U3) = C{vi)C{v2 - ui) + C{v2)C{v3 - vi) 

+ C{V3)C{VI~V2), 



i=l 

where gi{v), i = 1,2,3, are all continuous, even, nonnegative 
and integrable functions over the real line, which are non- 
increasing over [0,00). 

Remark 3. Condition 5 is satisfied by a Gaussian process, as 
the function P(-) reduces to the function Pg{')- 

Theorem 2. Under Conditions 1-5, the variance of the 
estimator </)„(•) given by f l/.2l ) converges as follows: 

/oo 
K\x)dx, 
-00 

where (5o,a is 1 if X — and is otherwise. The convergence 
is uniform over any closed and finite interval that does not 
include the frequency 0. In particular, the variance converges 
to 0. 

It follows from Theorems 1 and 2 that, under Conditions 
1-5, the estimator (/),i(A) is consistent, and is uniformly 
consistent over any closed and finite frequency interval that 
does not include the point 0. 

III. Rates of Convergence 

The rate of convergence of the variance of (/)„(A) follows 
from Theorem 2. We assume a few further conditions in order 
to arrive at a rate of convergence for its bias. These include 
additional conditions on the shapes of the covariance function 
and the kernel function. 

Condition lA. The function defined over the real line 

as hq{t) = sup^>|j| |s|''|C(s)| is integrable, for some positive 
number q greater than 1. 

Condition IB. The spectral density is such that, for some 

p > 1, 0(A) is 0{\X\-P), i.e., lim |A|P0(A) = A for some 

A—)- 00 

positive A. 



For any kernel A'(-), let us define 

k 



1 - K{x) 
lim — 



for each positive number r such that the limit exists. The 
characteristic exponent of the kernel is defined as the largest 
number r, such that the limit exists and is non-zero f26J. In 
other words, the characteristic exponent is the number r such 
that l-K{l/y) is O(y"0- 

Condition 2A. The characteristic exponent of the kernel K{-) 
is equal to the number q, for which Condition lA is assumed 
to hold. 
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Remark 4. Condition lA implies Condition 1 (see Remark 
1), and also that 4>{-) is [q] times differentiable, where [q] is 
the integer part of q. Thus, the number q indicates the degree 
of smoothness of the spectral density. If Condition lA holds 
for a particular value of q, then it would also hold for smaller 
values. 

Remark 5. The number p indicates the rate of decay of the 
spectral density. The following are two interesting situations, 
where Condition IB holds. 

1) The spectral density </){■) is a rational function, i.e., 
0(A) = ^[ij' where P(-) and Q{ ) are polynomials such 
that the degree of Q{-) is more than degree of P( ) by 
at least p. Note that continuous time ARMA processes 
possess rational power spectral density. 

2) The function C( ) has the following smoothness prop- 
erty: C(-) is p times differentiable and the p'^ derivative 
of C(-) is in L\ 

Remark 6. The number p can be increased indefinitely by 
continuous time low pass filtering with a cut off frequency 
larger than the maximum frequency of interest. There are 
well-known filters such as the Butterworth filter, which have 
polynomial rate of decay of the transfer function with specified 
degree of the polynomial, that can be used for this purpose. 

Theorem 3. Under Condition 2-4, lA, IB and 2A, the bias 
of the estimator (j)n{X) given by ( I/. 2 1 ) is 

i?[0„(A)-<^(A)] 



1 

^2^ 



(2^) 



-y- 

P \]\v 



\t\C{t)e-''^dt 
1 



Pn 



'Pn 

V n 



|/|>o 



(pn) 



1 



E[MX) - 0(A)] = O {{pnhnY) + O (^) + O (^j^ , 
uniformly in A over any closed and finite interval. 

Remark 7. Condition 2A can be relaxed to the extent that 
the characteristic exponent of the kernel K{-) is required to 
be greater than or equal to the number q, for which Condition 
lA is assumed to hold. If it is strictly greater than q, then 
the term O {{pnbnY) in the above theorem would have to be 
replaced by o {{pnbnY)- This follows from equation (IA.17b in 
the appendix and the fact that fc^ = in this case. On the 
other hand, if a kernel with characteristic exponent less than 
q is used, then one does not fully utilize the strength of the 
assumption on the smoothness of the spectral density, implied 
by Condition lA, and hence gets a slower rate of convergence. 



A. Choice of p„ and 6„ 

From Theorem 3 and Theorem 2, it is observed that the bias 
and the variance of the estimator 0„(A) converge to zero at 
different rates. We set out to choose the sampling rate p„ and 
the window width fe„ in order to ensure that MSB of (/),i(A) 
converges to zero as fast as possible. It would turn out that this 
happens when the squared bias and the variance go to zero at 
the same rate. 

Theorem 4. Under Conditions 2-5, lA, IB and 2A, the 
optimal rate of convergence of the MSE of the estimator 0„(A) 
is given by 



MSE[(I),,{\)] = O j , (III.l) 

which corresponds to the optimal choices 

(111.2) 
(III.3) 



Pn — P W+i+^Pi , 

P + 9 



and bn = Q n~ p+i+'^pi , 

for some positive constants P and Q. 

The above optimal rates of p„ and 6„ lead to the following 
corollaries to Theorems 2 and 3, respectively. 

Corollary 1. Under the Conditions 1, 2, 5 and the choices of 
Pn and bn given by ( l///.2l - [7ZOt , we have 

1 

lim nFF™V^ar[0(A)] = x)[cl){X)f / K^{x)dx, 

Q J-oo 

(111.4) 

where Sq^x is equal to 1 if X = 0, and is otherwise. 

Corollary 2. Under the Conditions lA, IB, 2, 2A and the 
choices of p„ and bn given by ( l///.2l - [7Z01 ), we have 



lim np+i+^pi E[(l)n{X) - 0(A)] 



(111.5) 

where the constant A is as in Condition IB. 

B. Comparison with Poisson Sampling 

Among the various schemes for sampling a continuous time 
stochastic process at irregular intervals, the Poisson sampling 
proposed by Silverman O is the simplest and most popular. 
Here, we compare the asymptotic behaviour of the estimator 
0„(-) with a similar estimator based on Poisson sampling. 

Let {ij}"^o be the sampling points from a Poisson process 
with average sampling rate p. Masry ||6l proved that, under 
Conditions 1, 2, 3 and 5, the estimator V'n(') defined as 



V^„(A) 



^ n—ln—i 

— Y,J2x{t,)X{t,+,)K{bn{t,+,~t,))cos{X{t,+,-t,)), 



npn 



i=i ]=i 
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is consistent for 4'{X) for any choice of p. 

Under the above conditions, the asymptotic variance of 
4>n{X) satisfies 

lim {nbn)Var[tljniX)] 

2 .00 (III.6) 

(1 + (5o,a)/ K^t)dt. 



C(0) 



2ttp 



For specifying the rate of convergence of the bias, Masry 
[|6l assumed the following additional conditions. 

Condition IC. \t\'^C{t) is integrable for some positive inte- 
ger q. 

Condition 2B. K{-) is q times differentiable with bounded 
derivatives, where q is an integer for which Condition IC 
holds. 

Note that Condition IC is implied by Condition 1 A with the 
same or higher value of q as is used here. Masry ||6l showed 
that, under Conditions 1, 2, 3, IC and 2B, the bias of the 
estimator ipn{X) is given as 

BiasiMX)] -i;[^„(A)] -0(A) 



9-1 

1=1 



(z)'K(')(0)b^, 



q^^'HX) + 0{bl) + 



(111.7) 



It follows from ( IIII.7b that the rate of convergence of the 

bias of ^„(A) is O (max n~^}), where 

iq if X(')(0) = for 1 < Z < 9, 

[ lo if XC") (0) ^ and (0) = for 1 < ^ < < q- 

The fastest possible rate of convergence is 
O (max {6^, n~^}), and this is achieved when one uses a 
kernel, which further satisfies Condition 2A with the same or 
higher value of q as is used here. In such a case, we have 



Bias[ib„{X)] = Oibl] 



0(1 

n 



(III.8) 



If the condition 2A holds with a higher value of q, then the 
term 0(&^) has to be replaced by 0(6^) 

Let us now assume that the kernel is chosen appropriately 
to ensure dlll.Sb . Note that the bias and the variance of 'ipn{X) 
converge to zero at different rates. One can choose the rate 
of convergence of the window width 6„ such that the MSB 
converges as fast as possible. It turns out that if 6„ = 0(n^"), 
then the optimal choice of a is ytI' which case the squared 
bias and variance of ipnW are both 0{n 25+1 ). 

In surmnary, under Conditions 1, IC, 2, 2A, 2B, 5 and 



R n 25+1 ^ 



(111.9) 



the MSB of TpniX) is 

MSE[MX)] 



I 2g 
O n 



The rate of convergence of MSB of (/)„(A) is given in The- 
orem 4 under Conditions lA, IB, 2, 2A, 5 and ( IIII.2tiIII.3l ). 
Both the results hold when p„ and 6„ for ipni-) are chosen 
as in ( IIII.2tilIL3b . 5„ for V'n( ) is chosen as in ( |III.9t and the 
following conditions hold simultaneously: Condition lA for 
some q greater than 1 (which implies Condition IC for [q] 
and Condition 1), Condition 2, Condition 2 A (for the same q 
as in Condition lA), condition 2B for [q], and Condition 5. 
Under this common set of conditions, we have 



MSE[iPn{X)] = O (^n" WT 
MSE[4)n{X)] = O (n~=^5+i+,/p 

When q is an integer, i.e., [q\ — q, the rate of convergence 
of the MSB of (/>n(A) is slower than that of ipn{X)- The two 
rates are comparable if p is much larger than q. When q is 
not an integer, the MSB of 0,1 (A) converges faster when p > 
Qiol/il ~ [9])' and in particular when p is very large. As we 
have indicated in Remark 6, for every fixed q, one can make 
p suitably large through low pass filtering. 

If the rates of convergence are comparable, the constants 
associated with these rates become important. We will compare 
the constants of the asymptotic bias of 0„(A) and tpniX) as 
well as the constants of their asymptotic variance separately, 
assuming that q is an integer. 

Under Conditions 1, 2, 5 and ( |III.91 l. we have 



lim n^i+^Var[ipn{X)] 



0(A) 



C(0) 



2ttp 



(l + So.x) K\x)dx. 



(in. 10) 



On the other hand, we have from Corollary 1 that under the 
Conditions 1, 2, 5 and ( |IlL2HlIL3] l. 



lim n29+i+5i/pyar[0„(A)] 

n— >ctt 

-^(1 + V)['^(A)]' 



K'^{x)dx. 



The ratio of the constants for the asymptotic variances of 

^pn{X) and 0„(A) is 



R' 



1 



C(0) 



(III. 11) 



27rp0(A)_ 

This ratio depends on the Poisson sampling rate p and the 
true value of the power spectral density 0(A). This ratio can 
be much larger than 1, particularly for larger values of A. In 
fact, even if p is chosen to minimize this ratio for a given 
value of 0(A) (though this is not practically possible), the 
minimum value happens to be 2QC(O)/[i?7r0(A)], which can 
be arbitrarily large for large values of A. Thus, the variance 
of V'rt(A) can generally be expected to be larger than that of 

0n(A). 

We now turn to the comparison of the expressions for bias. 
Under Conditions IC, 2, 2 A and 2B along with ( |III.9l l, we 
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have 

lim n^[E[$nW]-cbW] 

n— >-oo 

1 



2n 



(III. 12) 



On the other hand, we have from Corollary 2 that under the 
Conditions lA, IB, 2, 2A and (llll.2tilll.3b . 



lim np+i+^pi E[(j>n{\) - (/'(A)] 
= - (PQn,^ r WC{t)e 

271- J-oo 



itx 



dt 



1 A ^ 1 
(27r)P ^ mp' 



The first term of the expression on the right hand side is 
proportional to the expression on the right hand side of ( 1111.121 ). 
These terms are small for large values of A, while the second 
term of the expression on the right hand side of the above 
inequality does not depend on A. Even if the value of the 
second term is small, it would make a difference for large 
values of A. Consequently, V'„(A) can generally be expected 
to have a smaller bias than 0„(A). 

In summary, even though both (/in (A) and 7/'n(A) are consis- 
tent estimators under the stated conditions, there is a trade-off 
between (j!)„(A) and ^„(A) in terms of bias and variance. There 
is no clear order between the constants of the mean square 
errors of the two estimators. 

In order to examine the validity of the asymptotic results and 
the above comparisons for small samples, we turn to Monte 
Carlo simulations, reported in the next section. 

IV. Simulation study 

In this section, we shall present the simulation study of 
performance of the spectral density estimators based on regular 
and Poisson sampled data. We consider a continuous time 
autoregressive (AR(4)) process having the spectral density 

„2 



1 



where ai = 0.65, a2 ~ 0.75, = 0.85, a4 = 0.95 and 
cr = 1. 

A process {X{t), —oo<t< oo} having the above spectral 
density can be written as 



X{t) = / h{t-s)dW{s), 

J —QO 

where the impulse function h{-) is given by 

4 

/i(i) = ^c,e-"'*l[o,oo)(t) 

and the constants i = 1,...,4, are the solution of the 
following system of linear equations: 

h{0) = (0) = (0) ^ 0; (0) = 1, 

h^{0) being the jth derivative of h{-) evaluated at 0. 



In view of the above representation, we simulate the process 
{Xo{t)} given by 



Xoit) 



hit~s)dW{s) 



The process {Xo{t), Q < t < oo} is not a stationary process. 
However, as t becomes large, the variance of the difference 
between the processes {Xo(t)} and {X{t)} becomes small. 
We find out the value of t, say to, such that Var{XQ(t) — 
X{t)) < 10^^, and consider the path of the simulated process 
{Xo(i)} from to onwards. 

For estimation, we assume that the underlying power spec- 
tral density satisfies Condition lA with q = 2. According we 
use the Banning Kernel 

K{x) = ^(1 + cos(7ra;))l[_i4] (x), 
which has characteristic exponent 2. 

A. Finite sample performance of the estimator (/)„(A) 

Here, we consider the performance of (/)„(A) over the 
frequency range [0,7r/2]. We used the optimal choice of 
sampling rate developed in Section IIII-AI to generate regu- 
larly spaced samples of the process for sample sizes n = 
100, 1000 and 10000. We assume Condition lA with q = 2 
and Condition IB with p = 8 (both of which actually hold 
for the underlying power spectral density). For the above 
choices, the optimal powers of n for the sampling rate and the 
window width are pn oc n^^^^ and b„ oc n^^/^^. We choose 
Pn = ni/21 and &„ = in-5/2i. 

Figure 1 shows the average of the estimated power spectral 
density computed from 500 simulation runs, the empirically 
observed bias and variance, together with the true power 
spectral density and the theoretical (asymptotic) bias and 
variance, respectively, for the three samples sizes. 

From these figures, it can be observed that as the sample 
size goes from 100 to 10000, the empirical values of bias and 
variance get closer to the asymptotic results. Moreover, the 
theoretical (asymptotic) computations are quite comparable to 
the empirical values, even for sample size 100. 

B. Finite sample comparison of (A) with Poisson sampled 
estimator ^„(A) 

We generate Poisson sampled data with the average sam- 
pling rate p = 1 for sample sizes n — 100, 1000 and 
10000, and compute the estimator ^,i(A) on [0,7r/2]. Here, 
the optimal power of n for the window width is 6„ oc rt^^/^. 
We use bn — \n^^/^. 

Figure 2 shows the empirical bias, variance and MSE of the 
estimators 0„(A) and '(/'n(A) computed from 500 simulation 
runs, as a function of the frequency, for sample sizes 100, 
1000 and 10000. 
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Fig. 1. The average estimated power spectral density 0n(') (left column), the bias (middle colvmin) and the variance (right column) for sample sizes 100 
(top row), 1000 (middle row) and 10000 (bottom row). The solid and the dotted lines correspond to theoretical (asymptotic) and empirical values, respectively. 
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Fig. 2. The estimated bias of tlie power spectral density 4>n{-) and ^pn{■) (left column), the variance (middle column) and the MSE (in log scale, right column) 
for sample sizes 100 (top row), 1000 (middle row) and 10000 (bottom row). The solid and the dotted lines correspond to regular and Poisson samplings, 
respectively. 
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From these figures, it can be observed that the bias of 
is generally less than that of (j)n{') while the variance of tpn{-) 
is larger than that of 0„(-). The differences diminish with 
larger sample size. These patterns are in accordance with the 
large sample comparisons made in Section IIII-BI The MSE of 
'0n(-) is larger than that of the 4>n{-) for larger frequencies. The 
MSE is plotted in log-scale in order to highlight the fact that 
this quantity, in the case of ^„(-), levels off to a constant value 
for larger frequencies, while in the case of (f>n{-), it continues 
to decline. This difference in behaviour is in accordance with 
the variance expressions given in (IIII.4I) and (IIII.lOl l. 

V. Discussion 

In this paper, we have shown that the smoothed periodogram 
based on regularly spaced samples of a continuous time 
stationary stochastic process is consistent, under certain condi- 
tions, provided that the sampling rate increases appropriately 
as the sample size goes to infinity. We have also shown that, 
under the conditions used in the proofs, the estimators based on 
uniformly and non-uniformly spaced samples have about the 
same rates of convergence. Thus, our results remove a widely 
perceived theoretical deficiency of a popular spectral estimator 
based on regular sampling. 

It has been a common experience, both theoretically and 
empirically [22 1, that the smoothed periodogram estimator 
(II. Il l of a non-bandlimited power spectral density has less 
variance and more bias compared to the corresponding es- 
timator ipn{-) based on Poisson sampling. What the results 
of Section |lll] show is that, even though the new asymptotic 
results presented in this paper establish consistency of the 
smoothed periodogram (/)„(•) and the rates of convergence of 
the estimators <j>n{') and ')/'"(■) comparable, the constants 
for the first order approximations of the bias and variance of 
the two estimators exhibit the same type of trade off, i.e., the 
constant for the bias term is larger in the case of and the 

constant for the variance term is larger in the case of tpn{-)- 

The new asymptotic calculations provide a theoretical jus- 
tification of using the smoothed periodogram with common 
sense, even if the underlying power spectral density is not ban- 
dlimited. This common sense approach consists of appropriate 
filtering of the continuous time process followed by sampling 
at a suitably uniform rate. Remark 6 and Theorem 4 give 
guidelines for choosing a suitable filter and an appropriate 
sampling rate, respectively, which may be useful for practi- 
tioners. 

The simulation results reported in Section |IV] illustrate how 
one can choose an appropriate sampling rate for estimating 
the power spectral density, and obtain results in line with the 
theoretical results. Even though the underlying spectral density 
in this example is not band-limited, the estimator 0n(-) (based 
on uniformly spaced samples) is found to have smaller MSE 
than V'n(-) (based on Poisson samples) for larger frequencies. 



The reverse order holds for smaller frequencies. This shows 
that there is no clear dominance of one kind of sampling 
over another This finding for finite samples complements our 
asymptotic results. 

Our results do not take anything away from the vast lit- 
erature on spectrum estimation through irregularly sampled 
data. These methods may be quite appropriate when one 
does not have control over the sampling mechanism, when 
irregular sampling is logistically feasible and methodologically 
not limited, or when regular sampling have to be avoided for 
a specific reason (other than its perceived inconsistency). Fur- 
ther, an irregular sampling scheme such as Poisson sampling 
can be used where an estimator based on it is expected, either 
through theoretical analysis or through simulation studies, to 
have smaller MSE than the corresponding estimator based on 
regular sampling scheme. 

We have shown in Section IIII-BI how our theoretical re- 
sults can be used to compare uniform and Poisson sampling 
schemes. The results compiled there may be used to make 
further comparisons under different constraints. For example, 
if there is a limit to the maximum average sampling rate and/or 
the maximum sample size, one may make an optimal choice 
of the window width for fixed values of these two parameters, 
and then determine the corresponding MSE. In the case of 
(j>n{-), the optimal choice of the window width (for given 
sample size and sampling rate) is given by (|A.19t , while the 
choice in the case of ipn{-) can be derived similarly from (IIII.6I 1 
and ( IIII.7I 1 |6|. The best rates and constants achievable under 
the two sampling schemes, under appropriate constraints, may 
then be used to make a suitable choice of the sampling 
scheme. However, if there is a hard restriction on the minimum 
separation between two successive samples (rather than a 
restriction on the average sampling rate), then one cannot use 
Poisson sampling at all. In such cases, irregular sampling may 
be done according to a renewal process, with the inter-sample 
distance having a restricted probability distribution. Such a 
sampling scheme would not satisfy the sufficient condition for 
alias-free sampling given in Theorem 1 of 0. Thus, one may 
have to look further in search of a suitable estimator based on 
non-uniform sampling under such a restriction. 

The proven consistency of the smoothed periodogram opens 
up the possibility of establishing consistency of parametric 
estimators of the power spectral density of a continuous time 
process based on regularly spaced samples, by allowing the 
sampling rate together with the sample size to go to infin- 
ity. Such asymptotic calculations may potentially be used to 
justify and/or fine-tune multi-resolution methods of spectrum 
estimation ||2T1 . 

Appendix 

We denote by Ki{-) a function that bounds the covariance 
averaging kernel K{-) as in Condition 2. Further, we denote 

Ki{0) by M. 
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Proof of Theorem 1. We shall show that the bias of the 
estimator (/)n(A) given by jl.H converges to uniformly over 
[A/, Xu] for any A;, A„ such that A/ < A„. In order to compute 
the bias, we evaluate E[^n{v)]' 



E[%{v)]=E 



j + \v\ 

Pn 



(A.l) 



Therefore, we have 

I u| <n 

Consider the simple function Sn{-), defined over [A/, An] x 
(— oo, oo), by 



\v\<r 



Pn 



X l[-7rp„,7rp„](A)l^i^ ^l(i)- 

Observe that Sn{\t)dt ^ E[^„{\)]. 

Define the function over [A;, A„] x (— oo, oo), by 



zvr 



'it\ 



Observe that S{X,t)dt ~ (j){X) which is continuous. 

For any t G (—00,00), let w„(t) be the smallest in- 
teger greater than or equal to p„i. Note that the interval 
j^un^j_(t) ^ VrAt) ^ contains the point t and lim„^oo £^(£1 _ 
For sufficiently large n, we have from Conditions 3 and 4, 



Sn{X,t) =i- ( 1 



Pn n J \ Pn J \ Pn 



-i»)„(t)A 

e l[-7rp„,7rp„](A). 

Proving the uniform convergence of _Bms[(/)„(A)] over finite 
interval [A;, A„] amounts to proving 



Sn{X,t)dt^ / S{X,t)dt 



lim 



uniformly over [A;, Xu]- By virtue of the continuity of the 
limiting function, this in turn is equivalent to proving that 
1^00 '^n{X, t)dt converges continuously over this interval ll27l . 
i.e., for any sequence A„ — > A, 



lim 



SniXn,t)dt= / SiX,t)dt 



where A„, A G [A;, A„]. 

By continuity of the function 5'„(A, t) with respect to t and 
A, we have from Conditions 3 and 4, for any fixed t, 

lim |5„(A„,i)-5(A,t)H0. 



Note that from Conditions 1 and 2, we have the dominance 



|5„(A„,i)| <M 

\v\<7l 



C 



1.^ ^iW < Mho{t). 

\ Pn ' PTi J 



where /lo(') is the function described in Condition 1. Thus, by 
applying the dominated convergence theorem (DCT), we have 



lim / SniXn,t)dt= / S{X,t)dt. 
Hence, £'[^„(A)] — > ^(A) uniformly on [A/, A„]. 



□ 



Proof of Theorem 2. The estimator ^„(A), given by (|L2] ). 
can be written as 



n-l 



1 1 ' 

i(A) = 7„(0) H %{v)K{bnv) cos 

2npn ^ 



irpn 



v=l 



Therefore, 



Var[<j>n{X)] =h+ 2/2 (A) + /3(A), (A.2) 



where 

h - 



/2(A) = Cov 
/3(A) = Var 



n-l 



1 1 / Xv 

7^ 7«(0), Yjn{v)K{bnv) cos( — 

27rp„ npn ^ \pn 

^ 71— 1 

X! ^n{v)K{bnV) COS 



v=l 



Before we consider the convergence of the above three terms, 
we simplify the computation of Cov{jn{vi) ,jn{v2)) for non 
negative vi and V2- Note from Condition 5 that 

E[jn{vi)%{v2)] 



j=l 1=1 
n / \p, 



Pn J \ Pi 
Vl l-j l-j + V2 



•) 1 



Pn Pn Pn 



Vl,V2) C\ 



i\<r 



U + Vi\ fu-V2 



- J2 Un{u,V,,V2) C(—) C\ 
^ I I , \ Pn J 



\u\<_n 



Pn J \ Pn 

U \ ^( M + Wl — W2 
Pn 



- > Un[U, Vi,V2) Q[ , , 

n I V VPn Pn Pn J 



(A3) 
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where ui,W2) is a function with values between and 

1 defined as follows 

[/„(«, Ul,W2) 

f 

0, u< —n + 1^2, 

1_V2^ ^n^V2<u<Tam{Q,V2-vi), 

= < v^^-i' min(U, W2 — Wi j < It < niax(U, W2~^i); 

u > n — ui. 

(A.4) 



Therefore, by using ( lA.ll ) and ( IA.3l i. we have 

Coi; (7„(ui),7„(w2)) 

= - Un{u,Vi,V2)C 



i ^ C/„(U,«1,«2)C(^)C 

\u\<n 

- C/n(M,-yi,?;2)(3 

77 ^ / 



u \ u + — W2 

Pn 



Pn 

Vi —U —U + V2 



(A.5) 



Pn Pn Pn 



We now use this simplified form of Cov{jn{vi),^n{v2)) to 
establish the convergence of the three terms, /i, /2(A) and 

^3(A). 

Using ( |A.5t and Condition 5, /i is given as 



- 5] c/„(„,o,o)cf-y 



< 



+ i ^ C/„(m,0,0)Q ('o, 

E 



|u|<?^ 

2C(0) 
(27r)2np„ 



|u| <r 



— U — M 
1 , 51(0)32(0) 



(27r)2np„ E^^I^^^J^^- 

\u\<n 



As in Theorem 1, we can view J2\u\<n ^(p^) 
integral of the function s„(-) defined by 



— as the 



j|<n 



^ Ptl ' Pn' 



Since s„(i) — > |C(OI' ™d s„(t) < HqU) holds from Condi- 
tion 1, we get lim„-.oo E|„|<„ \c{f) \ ± = \C{t)\dt 
by applying the DCT as in Theorem 1, under Condition 4. A 
similar argument, together with Conditions 4 and 5, ensures 
that lim„^ooE|„|<„33 (^) ^ = \9-i{u)\du. Both the 
limiting integrals are finite. So nbnh as n ^ 00. 



Using jA.Si and Condition 5, the term /2(A) is given as 
1^2 (A) I 



<- 



^ n — 1 
1-2^2 E 



^ / t — J. 

2^E 

' " v=l 
^ n — 1 
2 „2 E/ 



v=l 
n-1 



27r2p 



«| <r 



- V c/„(m,o,«) c 



l-ul <r 







u + v 






Pn/ 




. Pn 


u ^ 




' u — v 






Pny 




. Pn 



i ^ [/„(., 0,.) of^,^,^ 

" ^ \ Pn Pn Pn 



K{h^v)\ 
K{b,,v)\ 
\KM\ 



< 



'TT^nbnPr 



E 



\u\ <n 

l5i(0).92(0)| 



Pn 



(^EV(MI&nj 

'n-1 

J2 \Ki.bnV)\br^ 



U \ 1 
Pn/ Pn 



r, 2 ^ I E 53 

\|n|<n / \v—l 

The last expression does not depend on A. An argument as in 
the case of Ii will show that lim„_j.oo X]"=i \K{bnv)\bn = 
Jp°° K{x)dx, under Conditions 2 and 3. The convergence of 
the other sums have already been discussed in connection with 
the term Ii. Hence, nbnhi^) — ?> as rt — ^ cx) uniformly for 
all A. 

Now we will consider /3(A). Using ( |A.5t , this term can be 
written as 

n— 1 n— 1 

/3(A) =^ E Cov[%{vi),%{v2)]K{bnVi)K{bnV2) 



vi—1 V2—I 



' \V\ \ ( XV2 

COS 

Pn / V Pn 

=/3l(A)+/32(A)+/33(A), 



where 



n-1 n-1 ^. ^ 

/31(A) =^ E K{b^Vi)K{bnV2)cOs(^ 



Pn Vi = l V2 = l 

(n-1) 



AwA / AW2' 
cos — 

PnJ \Pn 



- 2^ Un[U,Vl,V2) C\ \C' 



-(n-1) 
n — 1 n— 1 



Pn 



/32(A) Y E /^(&nt'l)/^(^^nt'2)cOs(— ) COS ( — 
Pn„, _i„, 1 VP"/ VPny 



•ui — 1 V2 — I 
(n-1) 



1 V"--^-' / \ / 



^i— — (n— 1) 



fin 



Pn 



and 

/33(A) 



= i E E/^(&nI'l)/f(6nI'2)cOs('^] COS 

" ^ \P« Pn " ' 



u——(n — l) 
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53 



U + V2 



From Conditions 2 and 5, 

|n6„/33(A)| 

n-1 n-1 (n-1) / \ / • 

By using a similar argument as in the case of Ii, we have 

n-1 n-1 



lim oi 



-("-1) 



— 52 — 53 

PnJ \PnJ \ Pn 



gi{vi)dvi 



g2{u)g3{u + V2)du 



dv-2 



Hence, n5„/33(A) as n ^ cx) uniformly for all A. 

Consider the term /31(A), let j^ui +W2, l~u — V2, i — V2 
/31(A) 

=^T^^f.^K^^K^)K{bn{J-^))cos(f)J^^'-'^ 

{n-l)-i 



Pn 



J2 Ur.il +hj-i,^)c(^-±^]c(- 



; = -(n-l)-i 

From Condition 2, observe that 



Pn 



nb„\l3i{X)\ 

-bn 



^-In-l+i (ri-l)-i 

^^^^T^E E E 

'^n i=l j=i+l Z=-(„-l)-i 



Pn ) \Pn 



Consider the simple function S'„, defined over (0, 00) x 

(0,00) X (—00,00) by 



Sn (2^7 ^ ) 
1 



i=l j=i+l /= — (n — 1) — i 

xl((i-i)6„.ib„](a^)l(i^ ^iW^itJ. J-]{t) 

^ Pn ' Pn' ^ Pn ' Pn' 

SO that 



oc poc poo 



nbn\hi{\)\ < 



Jo 



Snix,t,t )dxdtdt 



Since lim„^oo Pnbn = 0, we have, for any fixed (a;, t, t') G 
(0, 00) X (0, 00) X (—00, 00) and for large enough n, the 
inequality p„6„ < x/t, i.e., tpn < x/bn- Therefore, for large 
n, the unique integer j for which J-]{t) is non-zero is 

^ Pn ^ Pn ' 

smaller than the unique integer i for which (x) is 

non-zero. However, the ranges of summations in the definition 
of Sn{x,t,t ) do not permit the order j < i. Therefore, 
lining 00 Sn{x,t,t ) = 0. From Condition 1 and 2, we have 
the dominance 

\Sn{x,t,t)\ < M\Ki{x)ho{t + t)ho{t)\ (eL^ 



By applying the DCT, we have 

/ Sn{x,t,t )dxdtdt =0. 

Since the function Sn{x, t, t ) does not depend on A, we have 
nbnhiiX) — > uniformly for all A. 

In view of the convergence of the terms /i, /2(A), /33(A) 
and /31(A), we have 



lim nbn 



Var 



(mx)) -/32(A) 



= (A.6) 



uniformly for all A, and so we need to prove the convergence 
of nfe„ /32(A) only. 

Now consider /32(A) and let j = ui — W2, 1 — ^2, I — u. 

/32(A) 

=^E ^iz m,i)K{b^{i+j))cos{^) cosp('+^') 

Pn j^i j^_j^i 
(n-1) 



X- E Un{l,i+j.i)c(—]C\ 



n 



/=-(ri-l) 



P 

l+J 
Pn 



Pn 



(A.7) 



For A = 0, it follows from iA.6i and Lemma 1 below that 

/oo 
K^{x)dx (A.8) 
-oo 

For A ^ 0, we will further decompose /32(A) as follows. By 
applying the formula 2 cos(a) cos(6) = cos(a — fo) + cos(a + b) 
and cos(a + fe) = cos(a) cos(6) — sin(a) sin(6), we have 



/32(A) — /321(A) + /322(A) — /323(A), 



where 



/321(A) = 



1 1 



n—1 n— 1 — -i 



2^2„^2E E K{bn^)K{bn{^+3))cos\ 



(n-1) 



l=-{n-l) 



(A.9) 



n — 1 n—l — i 



-^322 (A) 

= iiE E ^(M/^(&n(*+j))cOs| 
^n j^i j=-i+i 

(n-1) / 7 \ /I -L ■ 

X J2 t^n(/,* + j,*)q- c i-^ 

1=^-1) VPn; V Pn 



and 

/323(A) 

^ ^ n — 1 n — 1 — ?' 



(hi) cos(—] 

\Pn) \Pn) 



(A.IO) 



27r2 np, 



"— " — -■-— ' / \ ■ \ / 

^E E /f(M/^(fen(*+j))sin — sin 

n ,=1 j=-r+l \PnJ \ 



Pn 



(n-1) / 7 \ // + ■ 

;=4;r-l) VPn; V Pn 



(A. 11) 
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It follows from equation iA.6h Lemma 2 and Lemma 3 
below that 



lim nbnVar[(j)n{X)] = lim ?t.6„/32i(A) 

/•oo 

= [0(A)]^ / K^{x)dx, 



(A.12) 



and the convergence is uniform over any closed interval that 
does not include the frequency 0. This completes the proof. 

□ 

Lemma 1. 

lim 7i5„ 132(0) 



1 



C{t + t )C{t ) dt 



dt dx. 



Proof of Lemma 1. Consider the simple function S'„( ), 
defined over (0, 00) x (— cx), 00) x (—00, 00) by 

Sn (a;, i ) 

^ n-l n-l~i 

= E E K{b,,t)K{br,{l+j))Un{l,l+j,t) 

1=1 j=-i+l i=-(n-l) 



Observe from ( IA.7l i that 

n6„/32(0) = 



00 />CXD />00 



J —00 J —00 



Snix, t, t )dxdtdt 



Define in{x), jn{t) and /„(t ) as the smallest integers 
greater than or equal to p„i and p„<', respectively. 

Thus, {X,t,t) £ {bnin-l{x),bnln{x)] X (^i-iiS,!^] X 

^] 5„z„(x) ^ X, ^ ^ ^ ^ as 
n — >■ 00. Since n6„ — > 00 and 6„p„ — as n ^> 00, we have, 
for any point {x,t,t ) G (0,oo) x (— cxd,oo) x (— oo,cxd) and 

large enough n, the inequalities —j-^ < t < "^''"^^ , i.e., 

— i„(x) < < n — in{x). Thus, for sufficiently large n, 

we have 



Snix,t,t ) ^-^K{bnin{x))K [ &„i„ (a;) + 6„p^ 



X [/„ (i ) , *n (a;) +jn {t),in {x)) 

Also, for large n, we have —{n—xbn) + y- <j^< [n—xbn), and 
so Un{ln{t),in{x)+jn{t),inix)) is positive and it converges 
to L Therefore, by virtue of Conditions 1 and 2, we have 

lim Sn{x,t,t) = ■^K^{x)C{t)C{t +t). 
Again, from Conditions 1 and 2, we have the dominance 

\Snix,t,t )\ < M\Ki{x)hoit + t )hoit )\ e L\ 



By applying the DCT, we have 



lim n6„/32(0) 

n—^oo 



- lim 

n— >-oo 



00 foo poo 



Sri{x^ t )dxdtdt' 



00 •/ —00 



C{t + t)C{t)dt 



dtdx. 



□ 



Lemma 2. The function /32i(-) converges as follows: 
lim n6„/32i(A) 



1 



cos(At) 



C(t + t)C{t)dt 



dt 



dx. 



The convergence is uniform on [A/,A,j] for arbitrary A/ and 
Xu such that A/ < A„ and A; An > 0. 

Proof of Lemma 2. Consider the simple function S'„( ), 
defined over [A;, A„] x (0, 00) x (—00, 00) x (—00, 00) by 

ri-l ri-l-i (n-l) 

S^{X,x,t,t)^—J2 E E K{br,t)K{b,,{i+j)) 
X cosf^) Ur.il, ^+J,^) c( -) c(^-±i) 

\PnJ \PnJ \ Pn J 

X l((i-l)6„,jb„](2;)lfiiii J_l(Ol(izii _Ll(i)5 

^ Pn Pn^ ^ Pn ' Pn ' 

SO that, from (|A.9t. 



n&n /321(A) 



00 /'OO /"CXD 



5'„(A, a;, t )dxdtdt 



CO J —QO 



A similar argument as in the proof of Lemma 1 will show that 

for (a;, t,t ) Q (0, 00) x (—00. 00) x {—00, 00) and sufficiently 
large n, 

Sn{X,X,t,t ) ^-^K{bnin{x)) k( bnin{x) + bnpj-^^^ 

\ Pn 



X COS 



i^^y^ Un{ln{t),in{x)+jn{t),in{x)) 



where i„(a;), jn{t) and Z„(t ) are the smallest integers greater 
than or equal to x/bn, pnt and pnt' , respectively, and that the 
function Sn{X,x,t,t ) converges to the function S{-), defined 
over (0,00) X (—00,00) X (—00,00) by 

S{X,x,t,t) = -^K'^{x)cos{Xt)C{t)C{t +t). 

Observe also that J^_^ S{X, x,t,t')dxdtdt' 

is a continuous function in A. As in the proof 
of Theorem 1, we prove the convergence of 

/o°°/^/^'^«(''^'^'*'^')'^^'^^'^*'™if°™l>' °" [Ai,A„], 
by showing that for any sequence A„ A, 



lim 



00 /'OO /"OO 



Sn{Xn,x,t,t )dxdtdt 

S{X, X, t, t )dxdtdt 



00 J —00 

00 /'OO /'OO 



00 J —00 
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for A„, A G [A/, Au]. The latter convergence follows, through We shall prove the convergence of nbnlz22{ ) given in ( |A.13l l 
Condition 1 and 2 and the DCT, from the dominance by proving the convergence of the two integrals on the right 



|5„(A,a;,t,t')| <M\Ki{x)ha{t + t )hQ{t)\ e L\ 



hand side of ( |A.14| i. 

In order to prove the first convergence, we follow the route 
and the convergence of the integrand, which holds because taken in Theorem 1, i.e., we show that for any sequence A„ —> A 
of the continuity of the kernel, the cosine and the covariance ^ ^ ^ 

function. lim / / / \Sn{^mX,t,t )—gn{Xn,x,t,t )\dxdtdt — 0, 



J-ooJ-oo 



Hence, nbnls2i ( ) converges as stated uniformly on [A;, A^ 

n for A, A„ e [A;, A^]. The above integral can be written as 

Lemma 3. The functions n6„J322(-) and nbnlz2'i{') converge 
to uniformly on [A; , A„] for arbitrary A; and A„ such that 
Xi < Xu and A/A„ > 0. 



CO /'OO /'C30 



\Sn{Xn,X,t,t ) - gniXn,X,t,t )\dxdtdt' 
< / \Sn{Xn,X,t,t ) - Gn{Xn,X,t,t )\dxdtdt' 



OO — OO 

OO /'OO poo 



Proof of Lemma 3. Consider the simple function Sn{-), 

defined over [A,, A„] X (0,oo) X (-00,00) X (-00,00) by ^ , - j^^^;^^^ ^'^ _ ^^^^^^ ^'^j^^^^^^/^ 



00 J —00 
00 /'OO /'OO 



— 00 — 00 



Sn{X,x,t,t) (A.15) 

ri-l ri-l+i / \ \ 

=— J2 K{bni)K{b„{i + j)) cosf — j where the function G„(-) is defined over [A;, A„] x (0, 00) x 

2^ *=i j=-»+ii=-(„-i) ^^"^ (-00, 00) X (-00, 00) by 

Mi^-l}bM(^) X =-i^cos(At)cos(^^Ji^ ) K\x)C{t)C{t + t)). 

Pn Pn Pn Pn ZTT y OfiPn / 



X 

SO that, from t IA.lOL 

poo poo poo 

n6n/322(A) = / / / Sn{X,x,t,t)dxdtdt. (A.13) 

A similar argument as in the proof of Lemma 1 will show that 
for (a;, t,t ) G (0, 00) x (—00, 00) x (—00, 00) and sufficiently 

l^ge where 



Now observe that 



\Sn{Xn, X,t,t ) — Gn{Xn, X,t,t )| 



f ^Xjitnixjbrji . , , 
COS a„(A„,a;,t,t ) 

V OnPn 



Sn{X, X,t,t 



an{Xn,X,t,t ) 



~Kib,Mx))K (b„i,,ix) +b„p^-^) cosf . /A^^\ , . 

271^ V Pn J \ Pn J ^K{bn%n{x))cOs[ ] Un{ln(.t ) , Jn{t) +ln(.x) , ln{x)) 

f2Xi (x)b A / \ Pn / 

^°'(, bnPn " j ),JnW +*nW,*„(2:)) / / l^jt ) + Ut) \ 



X C\ C\ 

Pn \ Pn 



pn \ Pn r - COs{Xnt) K{x)C{t)C{t+t). 

where i„(a;), j„(i) and /„(t ) ai-e the smallest integers greater Since q;„(A„, x, t') — > as n ^ 00, we have 
than or equal to x/bn, Pnt and p„t', respectively. 

For obtaining the uniform convergence of /322(')' consider n^oo ^' ^' ^ ) " G'„(A„, a;, t )| = 0. 



sup 

Ae[A,,A„] 



00 /"OO />oo 

/ / Sn{X,x,t,t )dxdtdt 

J —c<J —00 



00 />oo />oo 



Since from Condition 1 and 2, we have the dominance 

/ \Sn{XmX,t,t ) — Gn(Xn,X,t^t )\ 



< sup / / / \Sn{X,X,t,t)-gn{X,X,t,t)\dxdtdt ^ ^^TM^ ( \U I, ^^'W 

Ae[A,,A„]Jo J-ocJ-00 < 2i\/|ii:i(x)/io(i )/io(i + i )| e i 



- sup 

Ae[A,,A„] 



00 cOO cCfO 



gn{X, x,t,t )dxdtdt 

— 00 — 00 



(A. 14) By applying DCT, we have 

nOO /'OO 
/ \Sn{Xn,X,t,t')-Gn{Xn,X,t,t )\dxdtdt' = 0. 



(—00,00) X (—00,00) by 

27r2 y^bnPnJ y-^J^y- y^v'-T"'' Q^serve that for any fixed x, |x — i„(a;)6„| < 6„. By applying 



, 1 / 2Ax \ o ' ' Turning to the second term on the right hand side of ( IA.15I ), 

gn{X, x,t,t)^-^ cos(At) cos K^ix)G{t )Cit+t ). ^ ^ 



15 



the Mean Value Theorem to the cosine function in the interval 
" " w " Yve have 



COS 



sin(6') 



Pnbn 



for some 6* e 



cos 



PrJ^n Pnbn 

. Therefore 



PnK 



COS 



A. 



Pnb, 



< 



Thus, 



|G„(A„, x,t,t ) — (7„(A„, x,t,t )| 



<M^C^(0) 



An^n('^)^7) 
Pnbn 



Pnbn 



<M^C^(0) 



An 



So 



lim |G„(A„,x,t,t ) - gn{Xn,x,t,t )| ^ 0. 



From Condition 1 and 2, we have the dominance 

|G„(A„, X,t,t ) ~ (7n(An, X,t,t ) 

<2A-f|i^i(a;)/io(t + t')/io(i')| G i\ 

which leads us, through another use of the DCT, the conver- 
gence of the second integral of ( IA.15b . This establishes that 
the first term on the right hand side of (IA.14I I converges to 0. 
We only have to deal with the second term. 
Let 



s„(A) = 



OO fOO /'OO 



gn{X,x,t,t )dxdtdt . 



OO J —OO 



In order to establish the uniform convergence of s„( ) over 
[A/,A„], it is enough to show that s„(A„) — > for any 
sequence A„ A, where A, A„ e [A;,A„]. By using the 
Reimann-Lebesgue lemma, we have s„(A„) — > 0. Thus, the 
second term on the right hand side of ( IA.14b also converges 
to 0. Hence, ri&„/322(A) converges to uniformly on [A;, A„] 
as n — > OO. 

Convergence of nbnls-zsi^) to can be established in a 
similar manner. □ 

Proof of Theorem 3. Condition 1 ensures absolute summa- 
bility of the covariance sequence {C(/c/p„), fc S Z} of the 
regularly sampled process |x (^y-^ , t S for fixed p„. 
The corresponding spectral density ^„( ) is defined as 



en(A) 



1 ^ ^ J 

2npn \p 



e P" , As (— OO, oo). 



The function C„(-) is periodic with period 27rp„ and is related 
to the function (f>(-) as follows: 



en(A) 



OO 

E 

1— — 00 



(j){X + 2ttIp„), a e {-Trpn, T^Pn] 



In particular, for A e [— 7rp„, 7rp„], 

OO —1 

(/.(A) =e„(A) -^0(A + 2^Zp„) ~ ^ 0(A + 2^Zp„). 



1=1 



/ — — OO 



For sufficiently large n, irpn lies outside any finite interval 
[A/, A„], and the bias of the estimator (f>n{X) given by (II. 2b on 
[A/, A,j] can be decomposed as follows. 



e pt. — 



e p" — 



0(A) 
6. (A) 



+ ^ ?!)(A + 27r;/9„) + ^ 0(A + 27r/p„) 

/—I — cxj 

=Si(A) + B2(A) + B3(A) + S4(A) + B5(A), 



(A. 16) 



where 
-Bi(A) 



L-i:(l-A-(W)c(i)e-«, 



\v\<n 

^y:c'(-\e p. 



S2(A) 

S3 (A) 

B4(A) = ^0(A + 27rZp„), 
(=1 

-1 

^5 (A) = J2 <^(A + 27r/p„). 

/ — — 00 

We will consider each Bi{X), z — 1, . . . , 5, separately. 

1 



i?i(A) 

-E 



l-^s:(6„^;)^^ \v\i 

bl\v\i 



Pn \ Pn 



V \ _iv\ 1 

e Pi — 



Consider the simple function Sn defined over [A; , A„] x 

(— OO, oo) as 

Sn{X.t) 

]_ ^ fl-K{bnV)\ \V\1 



27r,^V b1\v\i 

\v\<r 



Pn \PnJ ^ P" 'P" 



Observe that (^)'Si(A) = /^^ ^„(A, t)dt. 

For any t E (—00,00), we define Vn{t) as the smallest 



integer greater than or equal to tp„. It follows that 



t as n 00, and for sufficiently large n and any t e (—00, 00), 
we can write 

1 ^l-j^(b„i;„(t))\ \Vn{t)\'^ ^fVn{t)\ 
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From Conditions 4 and 2A, we have 

lim Sn{t) = --Lfc |i|9C(t)e 
Also, Condition lA implies 

\Snit)\ < Mihg{t) where Mi = sup 
By applying the DCT, we have 



-itX 



1 - K{x) 



lim (^—] Si (A) = lim / Sn{X,t)dt 



'27r 



(A.17) 



Thus, -Bi(A) is 0{{pnbn)'^)- The fact that this convergence 
is uniform over the interval [A; , A„] can be established by 
choosing any sequence A„ in this interval that converges to 
A, and showing that Sn{\i,t)dt converges to the right 
hand side of ( IA.17I ). 

The term B2{X) can be written as 

/oo 
Sn{X,t)dt, 
-oo 

where S'„(-) is defined over [A;,A„] x (— cxd, oo) as 



27r ^ pn \Pn 

\v\<n 



As in the case of i?i(A), it can be shown that 

1 



lim 5„(A,t) 



2tt 



\t\Cit)e 



-itX 



From Condition lA, it follows that \Sn{\,t)\ < hq{t). Now 
again by applying the DCT, we have 

/oo 
Sn{X,t)dt 
-oo 

1 r°° 

= J Jt\Cit)e-^'^dt. 

Thus, B2{X) is O(^). The uniform convergence can be 
argued similarly as in the case of i3i(A). 

The term -B3(A) satisfies 



\v\>n 
'PnY 1 



\ n ) 2-K ^ \p„ 



\v\ >n 



n / Hn 



< 



(Pny I 

\ n J IT 



V>1 



Observe that for each fixed n, we have from Condition lA, 



III / 

lim ^ 



v — 1 ^ 



1 



< 



< lim 



Pn / Pn 



hg{t)dt < / hg{t)dt. 
^0 



So 



lim sup ^ 1 — 

n— )-oo _^ \Pn 



Pn 



< I hq{t)dt. 

Pn Jo 



Hence |i33(A)| is bounded by an 0{{^y) term, which 
converges to zero faster than B2{X). 

As for the term i?4(A), we have from Condition IB and 
DCT 



lim (pnTB^iX) 

, ^ {PnY 

= hm > t: — T—, — 

\X + 2nlpn\P 



A+27r?p„|P0(A + 27rZp„) 



1 °° 1 

— V lim \X + 2TTlpn\P(j){X + 2TTlpn)) 



1=1 

oo 



I 27rp„ 



y- 



(27r)? 
A 



Hence B4(A) = O y 

Similarly it can be proved that i?5(A) = O (■(^^fj^) • 

The theorem is proved by combining the five terms. □ 

Proof of Theorem 4. It follows from Theorems 2 and 3 that 
the MSB of the estimator can be written as 

MSE^^{X)\ - [£;{0„(A)-0(A)}]2+l/ar[0„(A)] 



o(w,.,-) + o(§)+o(') 



O 



nhn 



(A. 18) 



Let us first fix n and p„ and minimize the MSB with respect 
to 6„. The squared bias is an increasing functions of 6„, while 
the variance is a decreasing function of 6„. Therefore, the 
maximum possible value is minimized (i.e., the fastest rate 
of convergence is achieved) when (p„6„)^' oc (n5„)~^, i.e., 
when 

6„K(np29)-^. (A.19) 



By substituting this value in the expression for the MSB, and 

2c 
2q\ 



making use of the fact that < 2 and pnjn < 1, we have 



MSE[M>^)]=0[[^y'^' 



o 



pfF , 
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The first term on the right hand side is an increasing function 
of pn, while the second term is a decreasing function of p„. 
Therefore, the maximum of the two terms is minimized when 
(^) ^'^^ ^ Pn^^' i-S-' when p„ is chosen as in ( IIII.2b . The 
optimal rate for 6„, as given in ( IIII.3I ). is obtained by substi- 
tuting the expression for p„ in ( IA.19I ). Further substitution of 
these two optimal rates in ( IA.18I ) gives jllLlI ). □ 
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